Considering the Major Arguments against Random Assignment

Introduction Compared to most industrialized nations, the American system of primary and secondary education is uniquely decentralized. Federal, state and local authorities have a say in educational policy, with the state and local roles being most influential. As a result, educational goals and practices vary enormously, not only by state and school district, but also by schools within districts. To most foreigners, the American system must look like a cacophony of local experimentation. And it has probably come to look even more so in the last thirty years. Calls have emanated from politicians, business leaders and educational policy pundits seeking to improve schools through identifying effective school practices, setting higher standards, increasing accountability, developing more school-based management, founding charter schools, distributing school vouchers, creating smaller schools and class sizes, using more and newer technologies, and instituting more and better teacher and principal training. All this local experimentation indicates to some commentators a vibrantly democratic school system, free of the centralized control found elsewhere (e.g., Louis, 1998). But experimentation connotes more than implementing different ways of doing things. It also connotes systematically evaluating these alternatives-usually through deciding which alternatives are to be compared, which criteria they are to be compared on, how data on each criterion are to be collected, and how a decision about relative effectiveness is to be reached once all the alternatives have been compared on all the criteria. To scholars, experimentation further connotes: (1) studies that take place in laboratories from which all theoretically irrelevant causal forces have been excluded and in which the experimenter has well-nigh total control over when and how a causal agent is manipulated; or (2) using random assignment for determining which units are to be exposed to the 2 various treatment alternatives under test-in education these units are typically schools, classrooms or students. the purpose of such random assignment is to rule out the possibility that any observed post-assignment group differences are due to pre-existing group differences rather than to differences in the treatment experienced. Education uses experiments in both these senses. However, our focus on evaluating reforms in school settings inclines us to be concerned only with random assignment whose superiority for drawing causal inferences in non-laboratory settings is routinely acknowledged in the philosophy of science and in method texts in health, public health, agriculture, statistics, micro-economics, psychology and those parts of political science and sociology dealing with improving the assessment …